A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies

Active labor market policies aim to affect the outcomes – earnings, employment, health, etc. – of those who participate in them. The task of evaluation research lies in devising methods to reliably estimate their effects on those outcomes, so that informed decisions about program expansion and termination can be made. The past thirty years have witnessed real progress in our understanding of how to undertake evaluations of active labor market policies. The chapters by Heckman, LaLonde and Smith (1999) and Angrist and Krueger (1999) in the most recent Handbook of Labor Economics capture the rapid pace of ideas in this area and the lively intellectual debate it engenders. This paper lays out the basic form of the evaluation problem and then considers different methods for solving it. In describing the evaluation problem in Section 2, I highlight the role of individual heterogeneity in program impacts. Such heterogeneity has important implications both for the choice of impact estimator and for the interpretation of the resulting estimates. The remainder of the paper considers alternative methods recently advanced in the literature and employed in practice for evaluating active labor market policies. All of the methods I consider have been employed not just in the evaluation of active labor market policies, but also more broadly in the applied economics literature. I begin in Section 3 by considering social experiments, sometimes held up as the " gold standard " of evaluation. I clarify both the strengths and the weaknesses of experimental methods. I argue that experimentation represents an important evaluation tool that should neither be summarily dismissed nor uncritically accepted. 2 In Sections 4 and 5, I consider the two non-experimental methods most popular in the recent literature: difference-indifferences and propensity score matching. To keep the paper short, I leave a more general treatment of non-experimental evaluation methods emphasize that both the difference-indifferences and propensity score matching methods depend critically on maintained assumptions about the nature of the process by which participants select into a program. These assumptions may or may not hold empirically in any particular context; indeed, the findings in Heckman and Smith (1999) suggest that the assumptions underlying the difference-indifferences estimator represent a very poor approximation to reality in the case of job training programs. While how to choose among alternative non-experimental estimators remains an important issue, I conclude my discussion of partial equilibrium evaluation methods in Section 6 by arguing that …